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Dear Sir: 

The undersigned, Shrinath Acharya, hereby declares that: 

1 . I am one of the inventors of the subject matter described and claimed therein. 

2. The invention was made by us while we were employed by Margi Systems, Inc. 
("Margi"). The invention is related to the work we were employed to perform, and the invention 
was made within the scope of our employment duties. Prior to November 1, 2000, we conceived 
of the invention as described and claimed in the subject application in the U.S. as evidenced by 
the following: 

3. Attached as Exhibits A and B are relevant portion of documents (dates and certain 
sections of text have been redacted in the attached) created prior to November 1, 2000 setting 
forth various aspects of our invention. Exhibit A shows overall block diagrams of the system 
and an example of the sequence for processing an image for rendering on a display. Exhibit B 
includes portions of a draft of the application and describes various aspects of the invention. 
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4. Prior to November 1 , 2000, a working version of the invention as described and 
claimed in the subject application was made in the U.S. as evidenced by the following: 

5. Attached as Exhibit C is a relevant portion of schematics (dates have been 
redacted in the attached) created prior to November 1, 2000 setting forth various aspects of our 
invention. 

6. Exhibit C shows the schematics of a prototype designed for a HandSpring 
Personal Digital Assistant (PDA). The schematics depict the input/output connector, the power 
section, the display controller host interface, the display controller power and miscellaneous 
circuits, the CRT and DFP interface, the Spring Board connector, and the Host Display controller 
interface. The prototype depicted in Exhibit C included software that allowed for execution of 
the flow diagram embodied in Figure 2 of Exhibit A. The software was loaded into the 
prototype, tested, and performed satisfactorily. 

7. All statements made herein are of my own knowledge and are true and that all 
statements made on information and belief are believed to be true, and further that these 
statements are made with the knowledge that willful false statements and the like so made are 
punishable by fine or imprisonment, or both, under Section 1001 of Title 1 8 of the United States 
code, and that such willful statements may jeopardize the validity of the application or any patent 
issued therefrom. 

Respectfully submitted, 
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BACKGROUND 



Field Of Invention 



The present invention is related to handheld computing systems and, more 
15 particularly, to a method and apparatus for control and display of information on a 
display device using a handheld computing system. 



Copyright And Trademark Notice 

A portion of the disclosure of this patent document contains material which 
20 is subject to copyright protection. The owner has no objection to the facsimile 
reproduction by any one of the patent document or the patent disclosure, as it 
appears in the Patent and Trademark Office patent file or records, but otherwise 
reserves all copyrights whatsoever. 
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Certain marks referenced herein may be common law or registered 
trademarks of third parties affiliated or unaffiliated with the applicant or the 
assignee. Use of these marks is for the purpose of providing an enabling disclosure 
by way of example and shall not be construed to limit the scope of this invention to 
5 material associated with such marks. 

Related Art 

With the advent of the computer, application software has been implemented 
that allows a user to prepare a presentation in a digital form and later display it using 
a computer. As computers have become more affordable and laptop, digital 
10 presentations have become even more popular. Using a laptop computer, business 
people can conveniently produce, carry, and display their presentations as they move 
from one place to the next. Unfortunately, however, for a frequent business traveler, 
carrying the lightest of laptop computers still can be rather inconvenient in 
comparison to carrying a handheld computing system. 

15 

Commonly referred to as palmtops or personal digital assistants (PDAs), 

handheld computers are, typically, small enough to fit into a shirt pocket or on the 

palm of your hand. Unfortunately though, due to their small size, handheld 

f 

computers are less powerful than a typical laptop computer and therefore support 
20 only a limited number of applications. For example, while most handheld 

computers can support simple word processing applications, they do not support 
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graphic or presentation application such as Microsoft PowerPoint. Further, 
handheld computers lack the capability to connect to a display device or a power 
source that can support a long lasting presentation. 

5 It would be very useful for a business traveler to be able to conveniently use 

a light personal computing device to give a presentation. This invention, its 
advantages and improvements over the prior art schemes and devices, will be better 
understood and appreciated by reviewing the following discussion of the current 
portable computing systems utilized during presentations. 

10 

Personal Digital Assistants and Palmtop Computers 

With advancements in computer and communication technology, the size of 
computing devices has been dramatically reduced. Although the reduction in size 
compromises the power and functionality of a computing system, in recent years 
15 portable computers have been manufactured that have substantial computing power 
and utility. Moreover, the evolution of the portable computing devices has been 
consummated with the introduction of very popular and relatively powerful 
handheld computing devices commonly referred to as palm tops or personal digital 
assistants (PDAs). 
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A PDA, generally, refers to a handheld computer that serves as a digital tool 
for the storage and organization of personal information. It generally includes an 
information directory, a calendar, a to-do list, and some form of a conventional text 
processing application. Some PDAs are pen based and use a stylus to tap selections 
5 on graphic menus that are displayed on their screens. Some allow for a user to enter 
printed characters by simply writing on the surface of the display. A PDA may also 
include a small on-screen or off-screen keyboard which can be tapped with the 
stylus pen. Some PDAs provide for data synchronization between a PDA and a 
desktop computer via cable or wireless communication. A palmtop computer is like 
10 a PDA except that a palmtop typically includes a small keyboard whereas the PDA 
includes a pen. Apple Computer's Message Pad, more commonly known as the 
"Newton ®," was the first to popularize the concept of handheld computer devices. 

Due to the advancements in computing power and portability, currently 
15 manufactured handheld computing devices conveniently provide a user with access 
to a wide variety of features and services. A popular handheld computing device is 
the Palm Pilot by Palm Corporation, Santa Clara, California. A Palm Pilot can be 
used for general and specialized tasks. For example, a user can utilize it to schedule 
appointments, record contact information, browse the Internet, and send emails. The 
20 powerful and easy to use features of the Palm Pilot have made it an integral part of 
the daily life of many of its users. 
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Unfortunately, a short coming of the Palm Pilot and computing devices of 
the like is that they can only support a limited number of functions in comparison to 
their larger counterparts, such as laptop computers. Unlike laptop and desktop 
computers, the current handheld portable computing devices do not include mass 
5 media storage units (e.g., hard disk drives) or high capacity memory devices that 
allow for the storage, retrieval, and execution of one or more resource intensive 
software applications, such as the Microsoft PowerPoint®. As such, the handheld 
computers are limited to running customized software applications that are 
preloaded in their memory and can be supported by their limited resources. To 
10 overcome this problem, certain manufacturers (including 3Com) have developed 
handheld devices that include expansion slots for additional functionality. 

Handspring Corporation of Mountain View, California, a Palm OS licensee, 
manufactures the Visor®, a handheld computing device that includes an interface 

15 for the attachment of a separate module that enables the device with additional 

functionality over the conventional handheld computing devices. This interface also 
referred to as an expansion slot can be used to allow for the attachment of an 
electronic module that enables the device to act as a digital camera or a cellular 
telephone, for example. Unfortunately, no schemes or modules are currently 

20 available that allow a user to utilize a handheld computing device to display and 
control previously prepared digital presentations on a projector or other display 
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device. A method and apparatus that can provide for such functionality would be 
very useful. 

SUMMARY 

5 A method and system for control and display of information on an external 

display using a handheld computing device is described. Embodiments of the 
system include an expansion module configured to attach to a computing device 
such as a personal digital assistance (PDA). 



10 In embodiments of the system, application and system software are either 

preloaded on the computing device's system memory or are alternatively accessible 
through an external memory. The application software provides the user with 
special functions or utilities (e.g., calendar, address book, etc.). The system software 
manages the general operation of the computing device. The computing device 

15 includes a microcontroller, system memory, system display, and a communication 
interface. The communication interface is preferably in form of an expansion slot, 
configured to receive an expansion module. 



The expansion module includes a communication interface to operably 
20 connect the module to the computing device via the expansion slot. The module, in 
certain embodiments, also includes a memory, a communication interface, and an 
external display controller. 



-6- 



M-8754 US 
672646 vl 

The system and/or module memory can include control software and 
presentation data (e.g., PowerPoint presentations). Alternatively, presentation data 
can be included in an external memory device, such as flash memory, operably 
5 connected to the module via an external memory interface. Presentation data can be 
loaded onto the module's memory or the external memory from a desktop computer, 
for example. The external display controller may include a conversion table in 
special memory to reformat presentation data to displayable data on an external 
display. One or more presentations, including audio or video clips or movies maybe 
10 stored in system or module's memory. 

When the expansion module is connected to the computing device via the 
expansion slot, the computing device detects the presence of the module and 
recognizes expansion module's memory as an extension of the internal memory. 

15 Control software residing on the system and/or expansion module's memory is then 
executed by the computing system's microcontroller causing graphic control 
interfaces (e.g., icons, menus, etc.) to be displayed on the computing system's 
display screen. A user using the control interfaces can operate the control software 
to cause the computing device to retrieve and display presentation data stored in 

20 memory. 
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In embodiments of the system, presentation data is decoded by system 
microcontroller and is displayed via the external display controller on an external 
display device. A user can control the manner and order of display using the control 
interfaces on the computing system's screen. 



10 
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DETAILED DESCRIPTION 

The invention is directed to a system and method for displaying information 
on an external display device using a handheld computing system. In the following 
description, numerous specific details are set forth to provide a more thorough 
5 description of embodiments of the invention. It is apparent, however, to one skilled 
in the art that the invention may be practiced without some or all of these specific 
details or variations in said details. In some instances, certain well known features 
have not been described in detail so as not to obscure the more relevant aspects of 
the invention. 

10 System Architecture 

One or more embodiments of the invention are directed to an expansion 
module that attaches to a handheld computing system to transfer and display 
information such as PowerPoint™ presentation on an external display device. 

15 Figure 1 illustrates a block diagram of a handheld computing system 1 10 coupled to 
an expansion module 120. Expansion module 120 is in turn attached to an external 
display device 130. Handheld computing system 1 10 can be a personal digital 
assistant (PDA). A PDA is an organizer for personal information. It generally 
includes at least a name and address database, to do list, and note taker. PDA's are 

20 pen-based and use a stylist to tap selections on menus and to enter printed 

characters. Some PDA's include data synchronization options to synchronize data 
between the PDA and a desktop computer via cable or wireless transmission. Two 
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most popular PDA's are the Palmpilot™ and the Visor™ manufactured by Palm Inc. 
and Handspring Corporations, respectively. 

As shown the handheld computing system 110, includes a microcontroller 
5 101, system memory 102, user interface 105, I/O controller 107, communication 

interface 106, system display 108, and system bus 100. Handheld computing system 
110 also includes a clock that is not shown in Figure 1. In certain embodiments 
microcontroller 101, system memory 102, and communication interface 106, and 
system clock can be included in one single chip. The single chip design is 
10 developed and used in portable or handheld computing systems and other electronic 
products where miniature design and size are highly valuable and required. 

In one embodiment microcontroller 101 is a Dragon Ball-EZ CPU 16.58 
MHz manufactured by Motorola, for example. System memory 102 is, for example, 

15 2 MB or 8 MB and can be expanded to a 32 MB or more by attachment of an 

expansion module. In embodiments of the invention, the attachment of expansion 
module 120 serves to expand system memory to 4 or 10 MB as it includes additional 
2 MB of memory. Control software 122 for controlling the operation of the system 
and image data can be loaded onto system memory 102 and/or memory 121. In 

20 embodiments of the invention, the execution of control software causes the system 
to display graphical user interface for interacting and controlling the system on 
system display 108. System display 108 can be a 160X160 liquid crystal display 
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(LCD). User interface 105 is a graphical user interface. In embodiments of the 
invention user interface 105 is implemented as hardware in handheld computing 
system 110. System bus 100 is the communication medium used to transfer data 
and control signals among the components of handheld computing system 1 10. 

In accordance with an aspect of the invention, communication interface 106 
is a 68-pin connector expansion slot. An example of an expansion slot used in one 
or more embodiments of the system is a Springboard connector manufactured by 
Handspring Corporation. The connector provides access to a portion of 
microcontroller 101 address and data space so that handheld computing system 1 10 
can communicate with other devices. Essentially, communication interface 106 is 
an expansion of the system bus that, for example, includes a 24 bit address bus, a 16 
bit data bus, one or more read/write lines, and one or more chip select lines. 

The address bus is an internal channel from microcontroller 101 to memory 
102 across which data addresses are transmitted. The number of lines in the address 
bus determines the amount of memory that can be directly addressed as each line 
carries one bit of the address. For example, 20 address lines are used to address a 
maximum of 1,048,576 bytes, and 24 address lines can address 16,777,216 bytes. 
The data bus is an internal pathway across which data is transferred to and from the 
microcontroller 101. The read/write and chip select lines provide the means for 
input and output of information to and from handheld computing system 1 10. I/O 
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controller 107 handles input/output requests between handheld computing system 
1 10 and expansion module 120 by presenting signals on read/write or chip select 
lines. As such, via communication interface 106, control and data signals are 
communicated to expansion module 120. 

5 

Expansion module 120 includes memory 121, communication interface 123, 
and external display controller 127. Expansion module 120 is attached to external 
display device 130 through external display controller 127. User interface buttons 
131 and 132 can be implemented as graphic interfaces (e.g., displayed on system 
10 display 108) or hardware (e.g., physical buttons) in the system. As it is described in 
further detail below, using the user interface buttons 131 and 132, a user can control 
the display of presentation on external display device 130. 

Memory 121, in one embodiment of the invention, is flash ROM memory. 
Hash memory is a memory chip that can be re-written and holds its content without 
power. Hash memory is widely used for digital film and storage, in a variety of 
handheld devices. In embodiments of the invention, memory 121 is used for storage 
of graphical information such as PowerPoint Presentations. As explained earlier, in 
certain embodiments of the invention, memory 121 is recognized as an extension of 
system memory when expansion module 120 is attached to handheld computing 
device 1 10. As such, memory 121 can be used to store control software 122 as well 
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as image data, if needed. In accordance with one aspect of the system, memory 121 
can also be used to back up information stored on handheld computing system 110. 

Communication interface 123 can be implemented in the form of an 
application specific integrated circuit (ASIC) or a field programmable gate array 
(FPGA). An ASIC is a chip that is custom designed for a specific application rather 
than a general purpose chip such as a microprocessor. The use of ASIC's improves 
performance over general purpose CPUs because ASIC's are hardwired to do a 
specific job and do not incur the overhead of fetching and interpreting stored 
instructions. A field programmable gate array functions in the same manner as an 
ASIC, however, a FPGA is a programmable logic chip with a high density of 
programmable gates. The gates can be programmed to perform a certain function. 
In embodiments of the system, communication interface 123 is implemented in form 
of an ASIC or FPGA that can multiplex data bits decoded by microcontroller 101 
into a format that meets the requirements of external display controller 127. (e.g., 
multiplex 16 bit data to 32 bit data). 

External display controller 127 converts images decoded by microcontroller 
101 to electronic signals that are then forwarded to external display device 130. 
External display controller 127 determines, for example, the maximum resolution, 
the maximum refresh rate, and the number of colors that can be sent to the external 
display device 130. In embodiments of the system, external display controller 127 is 
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an MQ-200, MQ-Tenerife, or MQ-300 graphic controller, for example, 
manufactured by MediaQ Corporation, Santa Clara, California. MQ-200 is a highly 
integrated 2D accelerated graphics and display sub-system on a single chip. In 
certain embodiments, external display controller 127 includes an integrated frame 
5 buffer. The memory buffer acts as a storage area for image data before external 

display controller 127 processes the data to meet the configuration requirements of 
external display device 130. An MQ-200 has a 2 MB embedded memory buffer, for 
example. 

10 In embodiments of the invention, microcontroller 101 architecture includes a 

system bus 100 that supports a 24 bit address bus and a 16 bit data bus. In 
embodiments of the invention, where external display controller 127 is, for example 
a MQ-200, communication interface 123 is an ASIC or FPGA that acts as a 16 to 32 
bit multiplexor. Communication interface 123 converts 16 bit data, forwarded from 

15 handheld computing system 1 10, to 32 bit data, because MQ-200 requires a 32 bit 
data bus. External display device 130 can be an external monitor such as a Liquid 
Crystal Display (LCD) screen, a projector, a cathode ray tube (CRT) monitor, or 
other display device. 

20 Figure 2 illustrates a flow diagram of a method of transmitting and 

displaying information on an external display device, according to one or more 
embodiments of the system. In the following, embodiments of the invention are 
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described as applicable to storage and display of PowerPoint presentations using the 
expansion module 120 of the invention as attached to a handheld computing system 
110, such as a Visor or a Palmpilot. However, said description is provided by way 
of example and should not be construed as a limitation. The invention can be used 
5 to store and process any kind of information for display on any medium and system. 

System Software 

The system of this invention operates in conjunction with one or more 
software applications. One aspect of the system is directed to compression software 
10 configured to compress and transfer PowerPoint presentations, created using a 
general purpose computer (e.g., a personal computer), onto handheld computing 
system 110. Another aspect of the system is directed to control software configured 
to control the operation of handheld computing system 1 10 in conjunction with 
expansion module 120 for display of image data on external display 130. 

15 

FIG. 3A illustrates handheld computing system 1 10 attached to a desktop 
computer 310. Computer 310 can be used to create PowerPoint presentations, for 
example. In accordance with one aspect of the system, the compression software is 
implemented as a printer driver that accepts input data from PowerPoint application 
20 or other presentation software running on computer 310. A plug-in macro is 

installed on computer 310 for the PowerPoint application, in a manner well known 
in the art, so that the plug-in macro can call the compression software as if it is 
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writing to a printer driver. Referring to FIG. 2, at step 210, the macro is executed on 
computer 310 to export and compress selected PowerPoint presentations into one or 
more compressed image files. 
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10 

To display the graphic presentations on external display device 130, 
expansion module 120 is attached to handheld computing device 1 10 and external 
display device 130, as illustrated in FIGS. 1 and 3B. Expansion module 120 is 
attached to handheld computing system 1 10 via communication interfaces 106 (e.g., 
15 springboard expansion slot) and communication interface 123 (e.g., ASIC/FPGA). 
In certain embodiments, expansion module 120 is connected to external display 
device 130 via a graphic transmitter, for example. Further details on the type of data 
output generated by the transmitter is provided below. 

20 In embodiments of the invention, when expansion module 120 is attached 

handheld computing system 110, computing system 110 automatically powers up 
expansion module 120. Communication interface 106 detects the attachment of 
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expansion module 120 and forwards a notification (e.g., an interrupt signal) to 
microcontroller 101 to indicate the attachment of expansion module of 120. 
Handheld computing system 1 10 includes embedded system software (e.g., Basic 
Input Output System (BIOS) and/or Operating System (OS)) that takes over the 
5 control of the system, once it is turned on. System software includes an essential set 
of routines for testing handheld computing system 110 and preparing it for operation 
by querying a small memory bank (e.g., CMOS) for configuration settings on start 
up. System software sets up pointers (interrupt vectors) in system memory 102 to 
access those routines and then resets expansion module 120. 

10 

System software then registers expansion module 120 with handheld 
computing system 1 10. Depending on system implementation, memory 121 is used 
for the storage of control software 122 that is directed to a method of controlling the 
display of presentation data on external display device 130. At step 220, handheld 
15 computing system 110 installs and loads control software 122 on system memory 
102 and executes it. Alternatively, all or part of control software 122 can be loaded 
and executed on module memory 121. Further, in accordance with one aspect of the 
invention, at step 220, external display controller 127 is initialized and programmed 
to process presentation data for display on external display device 130. 

20 

External display controller 127 is a low power consumption graphics 
controller that converts images (e.g., characters, graphic patterns, etc.) from a bitmap 
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format, for example, into electronic signals. In one or more embodiments of the 
invention, external display controller 127 is an MQ-200 that includes a 
programmable timing generator and a programmable pixel serializer. To program 
display controller 127 certain registers included in display controller 127 are 
5 programmed in a well known manner by control software 122. For example, the 
resolution, color depth, refresh rate, and other parameters such as the display mode 
are set by control software 122. 

In embodiments of the invention, handheld computing device 110 is attached 
10 to display device 320 via an intelligent port that can query the device for 

configuration information so that control software 122 can set the display parameters 
accordingly. For example, a VESA Display Data Channel (VESA DDC) standard 
communications channel between the display adapter and monitor can be utilized to 
pass Extended Display Identification Data (EDID) from the monitor to the display 
15 adapter. EDID information describes a monitor's characteristics (vendor name, 
serial number, frequency range, etc.). 

According to one aspect of the invention, at step 224, execution of control 
software 122 causes the system to display control interfaces associated with image 
20 data on system display 108. Depending on implementation and design, various 

control menus are displayed in form of a graphical user interface illustrated in FIGS. 
4 A through 4J. FIGS. 4 A through 4J are provided by way of example only and for 
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illustration purposes. Other embodiments of the invention can be implemented to 
provide user interactive menus with other look and feel. 



5 



10 



Once the selected file has been imported, the user can select the presentation 
by, for example, highlighting the name of the presentation (e.g., IBM.MRG). The 
import feature can be also used to import slides from another database into a 

15 currently selected database. FIG. 4D illustrates the graphical user interface 

displayed once a database is selected. Rectangles one through nine represent slides 
that make up the selected presentation. The slides can be also graphically presented 
in form of a list or other suitable manner. A user can select each individual slide for 
inclusion in a slide show. Alternatively, if the system is in the import mode, then 

20 the user can select one or more slide for inclusion in a currently selected database. 
That is, using the import feature, one or more slides in one presentation can be 
included in another presentation. 

-22- 



M-8754 US 
672646 vl 



In embodiments of the invention, as illustrated in FIG. 4E, a check box is 
associated and displayed next to each slide. A user may check the check box 
associated with a slide, if he wishes to include that slide in the slide show. The user 
5 can also rearrange the display order of the slide in the show by dragging and 

dropping the slide in a desired location. The user can view a slide in more detail by 
selecting a specific slide. FIG. 4F is an example of various information that may be 
displayed as a result of selecting an individual slide. For example, the slide's 
graphic features, title, and notes associated with the slide may be displayed. 

10 

Graphically displayed control buttons 440, 442, and 444 is illustrated in FIG. 
4E, can be used to control the manner in which slides are displayed. For example, 
buttons 440 and 442 may be used to advance the slide show forward or backwards. 
Button 444 may be used, for example, to start a slide show. In some embodiments 
15 of the invention, part of the control interface is implemented as hardware buttons 
131 and 132 as shown in FIG. 1. Thus, for example, pressing on button 131 causes 
the system to display a previous slide and pressing on button 132 causes the system 
to display the next slide. 

20 In addition to the above control buttons, a user may interact with drop down 

menus to control the display of a slide show. As illustrated in FIG. 4G, a user may 
select from a preference menu, for example, to start a manual or automatic slide 
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show. A manual slide show option is set when the user desires to control the 
transition from one slide to the next. If the user selects the automatic slide show, 
then as illustrated in FIG. 4H the system prompts the user to set the time transition 
between slides. In accordance with an aspect of the system, the user can for 
5 example set the time between 1 to 99 seconds, minutes, or a combination of the two. 
The user can also select to continuously display the presentation in a loop such that 
the presentation starts over once the last slide is displayed. 

FIG. 41 illustrates the user interface displayed on system display 408 while 
10 the slide show is in progress, during a manual slide show. Control interfaces are 

provided to advance the slide show forward or backward, or to stop the show. FIG. 
4J illustrates the user interface displayed on system display 408 while the slide show 
is in progress, during an automatic slide show. Control interfaces are provided to 
pause, continue, or stop the slide show. 

15 

Referring back to FIG. 2, at step 230, in response to user interaction with the 
system, control software 122 causes microcontroller 101 to decode and decompress 
image data associated with a selected PowerPoint presentation. In embodiments of 
the invention, decoded image information is forwarded to external display controller 
20 127, at step 232, via communication interfaces 106 and 123. In accordance to one 
aspect of the invention, data transfer between communication interfaces 106 and 123 
is accomplished by memory mapping. Memory mapping refers to a method of 

-24- 



M-8754 US 
672646 vl 

communication wherein, instead of directly writing to a communication port (e.g., 
I/O port) using standard read/write commands (e.g., IOR or IOW signals), specific 
memory locations are allocated for input and output functions. Thus, instead of 
presenting data using standard read/write commands, the data is presented to a 
5 designated memory location when memory read (e.g., MEMR) or memory write 
(e.g., MEMW) signals are activated. 



In certain embodiments, display controller 127 includes an on-chip frame 
buffer. An on-chip frame buffer is a storage medium (e.g., 2 MB DRAM memory) 

10 which is used to store image data decoded by microcontroller 101. FIG. 5A 
illustrates a memory mapped address space, in accordance with one or more 
embodiments of the system. The upper memory block (0x28000000 - 0x28 1FFFFF) 
represents 2 MB flash ROM memory 121 in expansion module 120, for example. 
The middle memory block (0x00000000 - OXOOlfOOOO) represents 2 MB system 

15 memory 102 in handheld computing system 110, for example. The lower memory 
block (0x29000000 - 0x291ffffe) represents 2 MB of DRAM in external display 
controller 127, for example. In embodiments of the invention, the exemplary 
memory space described above is mapped in a well known manner to allow 
microcontroller 101 to address both memory 121 and memory buffer in the external 

20 display controller as an extension of system bus 100. 
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5 



10 



15 At step 234, image data in the frame buffer is processed and an image is 

rendered on external display 130, while image information for the next frame is 
decoded by microcontroller 101. Rendering is the process of drawing lines, 
windows, characters, and other objects, in the frame buffer. Particularly, external 
display controller 127 scans the data in the frame buffer and outputs image data at a 

20 pre-configured horizontal or vertical sync rates to an analog or parallel digital output 
in a well known manner. External display controller 127 determines the maximum 
display resolution, refresh rate (e.g., 60 frames per second), and number of colors for 
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external display device 130. The following display resolution formats are examples 
of resolutions supported by one or more embodiments of the system: 

1024x768 (XGA) 8-bit color 
800x600 (SVGA) 8-bit color 
640x480 (VGA) 8-bit color 

In one or more embodiments of the invention, external display controller 127 
is an MQ-200 that includes one or more graphic controllers. Each graphic controller 
includes a programmable timing generator and a programmable pixel serializes The 
programmable timing generator generates timing signals that help establish a stable 
display area on external display device 130. The programmable pixel serializer 
fetches presentation data from the frame buffer, adds color information as needed, 
and forwards a stream of pixels to external display device 1 30. Presentation data is 
stored in the frame buffer as an array of pixels. Each pixel specifies the color for 
one point on external display device 1 30. In one or more embodiments of the 
invention, each pixel value corresponds to a 24-bit value stored in a color lookup 
table (LUT). The LUT includes one or more 256x8 tables, for example, expanding 
the pixel value to 24-bit color, with 8 bits each for red, green, and blue. 

In embodiments of the invention, the frame buffer is a first in first out 
(FIFO) memory buffer such that graphic controllers read image data into the frame 
buffer before it is needed to be displayed. Pixel data is then taken from the frame 
buffer one pixel at a time and input to the color lookup table for expansion into a 24- 
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bit color value, for example. The 24-bit color value is then input to the selected 
display interface for output to external display device 130, as discussed in further 
detail below. FIG. 5B illustrates various formats in which image data can be stored 
in the frame buffer, in accordance with one or more aspects of the invention. For 
example, image data can be stored as 8, 16, or 24-bit color bit maps. The outline of 
the storage of the image data (e.g., pixels) is shown in FIG. 5B. 



In the following, other embodiments of the invention are described in 
conjunction with one or more additional components that provide the system with 
additional functionality. Certain components are described with reference to 
specifications of certain manufacturers. These specifications are provided, however, 
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by way of example and should not be construed to limit the scope of the invention to 
these specific products. 

Graphic Transmitter 

5 In accordance with one aspect of the invention, expansion module 120 

includes a graphic transmitter 126 for converting parallel digital output generated by 
external display controller 127 to serial digital output for display devices, according 
to industry standards. For example, in one embodiment of the system, graphic 
transmitter 126 is a PanelLink® transmitter manufactured by Silicon Image Inc., 

10 Sunny Vale, California. As illustrated in FIG. 6, transmitter 126 resides in the 

expansion module 120 and receives the output stream generated by external display 
controller 127. In certain embodiments, transmitter 126 takes the digital stream 
generated by expansion module 120 and converts it to DVI-compliant digital output. 
DVI or Digital Visual Interface is a digital flat panel interface from the Digital 

15 Display Working Group (DDWG) that provides for a universal standard for sending 
digital information to display devices. 

Embodiments of the system support Low Voltage Differential Signaling 
(LVDS) and Transmission Minimized Differential Signaling (TMDS) transmission 
20 methods. These methods are well known and are typically used for sending digital 
information to a flat panel display. The advantage of using these transmission 
methods is that fewer wires need to be used for transmission of image data as 
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parallel digital output is converted into serial digital output. Furthermore, the above 
transmissions methods are more efficient in that they provided for lower power 
consumption, better protection against Electro Magnetic Interference (EMI), and 
higher transmission speeds for High Definition (HD) resolution signals. 

5 External Display Plug and Display Feature 

Certain embodiments of the invention, take advantage of a communication 
standard known as Display Data Channel or DDC to automatically configure the 
output format for external display device 130 upon attachment. A DDC is a 

10 standard communications channel that is used to pass Extended Display 

Identification Data (EDID) from external display device 130 to external display 
controller 127. EDID is stored in the external display device 130 and describes its 
characteristics (vendor name, serial number, frequency range, etc.). As such, DDC 
technology can be utilized to auto-configure the output generated by external display 

15 controller 127 so that the output is compatible with the requirements of external 
display device 130. Using DDC technology, external display controller 127 can 
query external display device 130 for EDID data, and configure the refresh rate and 
resolution of the output signals to match that of external display device 130 based on 
said EDID data. 

20 

Typically, DDC requires an additional wire in the connector cable that 
attaches external display controller 127 and external display device 130. The first 
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level implementation (DDC1) provides a unidirectional channel that lets the external 
display device 130 inform external display controller 127 of its characteristics. A 
second bi-directional level (DDC2) allows the external display controller 127 to 
adjust the external display device 130. In some embodiments of the invention, the 
5 external display device 130's switch settings can be also implemented into a 
software control panel, for example, using the DDC communication standard. 

External Microcontroller 

As illustrated in FIGS. 6 and 7 A through 7B, in one embodiment of the 
10 system, expansion module 120 includes a microcontroller 125. In this embodiment, 
microcontroller 125 takes over the control of external display controller 127 instead 
of microcontroller 101. Microcontroller 125 can be an on-board microcontroller or 
Digital Signal Processor (DSP) (e.g., a RISC microcontroller) in communication 
with portable computing system 110 through communication interface 123. In 
15 certain embodiments, micrcontroller 125 includes an on-chip memory 720 (e.g., 4k 
of Random Access Memory (RAM)) for storing program instruction and executing 
said instructions. Microcontroller 125 is implemented to execute program code 
without directly fetching program instructions from system memory 102 or module 
memory 121, thereby reducing complex routing and signaling requirements between 
20 handheld computing system 1 10 and expansion module 120, as described in more 
detail below. 
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In one embodiment, program code for processing and decoding image data is 
transferred via communication interface 123 to microcontroller 125 one word at a 
time. Microcontroller 125 saves each word of the program code in a specific 
location on on-chip memory 720. Once all the program code has been stored on on- 
5 chip memory 720 then the program code is executed locally in on-chip memory 720. 
Once microcontroller 125 starts executing the program code, then image data is 
transferred to microcontroller 125 so it can be processed. The program code running 
on-chip memory decodes and decompresses image data. 

10 In one embodiment of the system, the full width of data buses of handheld 

computing system 1 10 and microcontroller 125 and a minimum number of address 
lines (e.g., four address lines) are assigned to communication interface 123, in a well 
known manner. FIG. 5C illustrates a register map for register addresses 
implemented to provide four address lines in communication interface 123, in 

15 accordance with one or more embodiments of the system. This implementation 
obviates the need for providing microcontroller 125 with access to address buses 
associated with system memory 102 and/or memory 121. It further resolves the 
complications associated with signal routing and control arbitration issues between 
handheld computing system 110 and expansion modules 120, and other complexities 

20 that may arise from implementing the communication interface between the two. 
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As described earlier, in embodiments of the invention communication 
interface 123 is an ASIC or FPGA implemented to convert 16 bit data to 32 bit to 
meet the addressing requirements of expansion module 120. In addition to data and 
address lines, control lines are implemented between the following, in a well known 
5 manner: handheld computing device 110 and communication interface 123; 

communication interface 123 and microcontroller 125; and microcontroller 125 and 
external display controller 127. Microcontroller 125 is connected to external display 
controller 127 in a well known manner so that it can transmit decoded image 
information to external display controller 127's frame buffer. In this embodiment, 
10 handheld computing system 1 10's microcontroller 101 does not access external 
display controller 127. 

FIGS. 7 A through 7D are block diagrams according to one or more 
embodiments of the invention illustrating the manner in which program instructions 

15 are loaded onto on-chip memory 720. Communication interface 123 includes a data 
transaction register (DTR) 710. Data transferred from handheld computing system 
1 10 is stored data transaction register until it is processed by microcontroller 125. In 
accordance with one aspect of the invention, because microcontroller 125 cannot 
directly access program instruction and data stored in system memory 102 or 

20 expansion module memory 121, said information is fetched by microcontroller 101 
and transferred to microcontroller 125 via communication interface 123. Program 
instructions are loaded into data transaction register 710. Microcontroller 125 
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fetches the content of data transaction register 710 and stores it as instructions in on- 
chip memory 720 for future execution. 

FIG. 8 includes a flow diagram that illustrates a method for transfer of 
5 program code from handheld computing system 1 10 to expansion module 120's 
microprocessor 125, according to one or more embodiments of the invention. At 
step 805, system software loaded on handheld computing system 110 resets 
microcontroller 125. FIG. 7 A illustrates the contents of the data transaction register 
710 and on-chip memory 720 after reset. After reset, microcontroller 125 initiates a 
10 read from data transaction register 710, at step 810. At step 815, communication 
interface 123 is implemented to signal a wait to microcontroller 125. At step 820, 
microcontroller 125 waits for a release signal from communication interface 123. At 
step 825, communication interface 123 requests to receive data from handheld 
computing system 110. 

15 

At step 830, system software causes microcontroller 102 to fetch program 
code (e.g., code required for decoding image data, or any other program code) from 
memory 102 or 121. As described earlier to reduce routing and signaling 
complexities, microcontroller 125 cannot access memory 102 and 121 to address the 
20 program code or data stored thereon, directly. Therefore, at step 835, 

microcontroller 101 places the program code embedded in an instruction on the data 
bus and transfers the instruction to communication interface 123. At step 840, 
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depending on implementation, all or parts of the instruction are stored in data 
transaction register 710. 



In embodiments of the system, each instruction is 48 bits (i.e., contains three 
words) and data transaction register 710 is a 16 bit register. Because of its limited 
size, data transaction register 710 stores one word (each word is 16 bits) at a time. 
As a result, to process each instruction, data transaction register has to load three 
words, in three separate intervals. Each word in the instruction contains one of 
program code, data, and/or a function. The function instructs micro controller 125 
what to do with the data or program code stored in data transaction register 710. For 
example, as illustrated in FIG. 7A, the first instruction can be illustrated in the 
following manner: 



Load Imd 



Xbinsl 



StrXBAdl 



"Load Imd" is a load immediate function that includes an op-code indicating 
the type of instruction and the register address in which the next word loaded into 
transaction data register 710 is to be stored. In embodiments of the invention, 
microcontroller 720 in addition to on-chip memory 720, also includes another set of 
registers (e.g. eight registers) that are used for temporary storage of information 
before the information is stored onto on-chip memory 720. Xbinsl is a part of 
program code (e.g., image decoding software), for example, that is to be stored in 
on-chip memory for execution. Str XBAdl is the memory address of the location in 
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which Xbinsl will be stored. In accordance to one aspect of the invention, content 
of transaction data register 720, at step 840, can be illustrated as the following: 

Load Imd 



5 



10 



Xbinsl 

15 The second word (e.g., XBinsl) includes program code. At step 860, 

microcontroller 125, temporarily stores the second word in the register addressed by 
the load immediate function. At step 865, the third word (e.g., Str XBAdl) is loaded 
into transaction data register 710. In accordance to one aspect of the invention, 
content of transaction data register 720, after step 865, can be illustrated as the 

20 following: 
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Str XBAdl 

The third word (e.g., Str XBAdl) includes an address in on-chip memory 
720, where program instruction contained in the second word is to be stored. At step 
870, microcontroller 125 stores program code XBinsl in memory location XBAdl. 
5 FIG. 7B illustrates the contents of the data transaction register 710 and on-chip 
memory 720 after the step 870 is completed. 

It should be noted that the above described architecture is provided by way 
of example. Other embodiments of the system may be implemented to support 32 
10 bit, or 48 bit registers that can process an entire instruction line at one time. Other 
implementation are also possible. 

In accordance with one aspect of the system, after the first instruction is 
fetched, the system reverts to step 810 and initiates another read to fetch another set 

15 of instructions. FIG. 7C illustrates the contents of the data transaction register 710 
and on-chip memory 720 after the second set of instructions is fetched. This process 
continues, until the system reads and detects a jump function at steps 850 and 853, 
respectively. A jump function includes the address of the memory location in which 
the first set of program code is stored in on-chip memory 720 (e.g., XBAdl). Upon 

20 reading the jump instruction, at step 875, microcontroller 125 executes the program 
code stored in on-chip memory 720. FIG. 7D illustrates the contents of the data 
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transaction register 710 and on-chip memory 720 after the last set of instructions is 
fetched. In embodiments of the invention, the program code being executed in on- 
chip memory, for example, decodes image data fetched from system or module 
memory, at step 880. 

5 

Image data is fetched and transferred to microcontroller 125 for processing in 
a manner illustrated in FIG. 9. At step 900, microcontroller 125 initiates a read from 
transaction data register 710. At step 910, communication interface 123 signals a 
wait to microcontroller 125. At step 920, microcontroller 125 waits for the release 
10 signal. At step 930, communication interface 123 submits request to receive data 
from handheld computing system 110. At step 940, handheld computing system 
1 10's microcontroller 101 fetches data from system memory or module memory. 



At step 950, the data is transferred to expansion module 120's 
15 communication interface 123. At step 960, the data is stored in data transaction 

register 710 in communication interface 123. At step 970, communication interface 
releases the wait signal for microcontroller 125. At step 980, microcontroller 125 
reads the content of data transaction register 710. At step 990, the program code 
executing in on-chip memory 720 decodes data read from data transaction register 
20 7 10. At step 995, if more image data is to be decoded then the system reverts to step 
900. 



-38- 



M-8754 US 
672646 vl 

In accordance with one aspect of the system, microcontroller 101 transfers 
image data and program code to microcontroller 125's internal memory. As such, 
embodiments of the system that are implemented to include microcontroller 125 can 
operate independent from handheld computing unit 110. 

5 Wireless Communication 

Referring to FIG. 6, embodiments of the system are implemented such that 
handheld computing system 110, expansion module 120, and external display device 
130 can communicate in a wireless environment. Many handheld computing 

10 systems such as Palm and Visor are equipped with infrared ports for data 

transmission. Accordingly, embodiments of the invention are implemented, in a 
well known manner, to include infrared receivers and transmitters to communicate 
with the currently available devices. IrDA is a standard developed by Infrared Data 
Association, Walnut Creek, CA, that provides for infrared communication between 

15 various computing systems, for example. With IrDA ports, a handheld computing 
system can exchange data with a desktop computer or use a printer without a cable 
connection. IrDA requires line-of-sight transmission. 

In alternate embodiments of the invention, wireless connections can be 
20 established by incorporating communication interfaces that promote the exchange of 
information using radio (RF) technology, such as Bluetooth. Bluetooth is a wireless 
personal area network (PAN) technology from the Bluetooth Special Interest Group 
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(founded by Ericsson, IBM, Intel, Nokia and Toshiba). Bluetooth is an open 
standard for short-range transmission of digital voice and data between mobile 
devices (laptops, PDAs, phones) and desktop devices. Bluetooth provides up to 720 
Kbps data transfer within a range of 10 meters and up to 100 meters with a power 
5 boost. Unlike IrDA which requires a line-of-sight transmission, Bluetooth uses 
omnidirectional radio waves that can transmit through walls and other non-metal 
barriers. 

Other wireless standards and technologies, such as IEEE 802.1 1, may be 
10 used in other embodiments of their invention. 

Additional Features 

Referring to FIG. 6, embodiments of the system are implemented to include 
an external memory interface, so that an external memory module 140 can be 
15 connected to the system. External memory 140 can be a flash card or memory stick 
device. To maintain compatibility with VGA analog displays and TV set, an analog 
output and TV output may be also included to one embodiment of the invention, in a 
well known manner. 

20 Certain embodiments of the invention are implemented to include audio 

ports for attachment to microphones or speakers. In yet other embodiments of the 
system, an on/off switch 133 may be implemented to allow a user to turn expansion 
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module 120 on or off. Some embodiments of the invention may include a DC inlet 
for attachment to a power socket or a power feedback line, as illustrated in FIG. 6, 
for attachment to external display device 130, to conserve battery life of handheld 
computing system 110. 

One or more embodiments of the invention are also implemented to work 
with a remote control device. According, a user may use a remote control device to 
control the display of images on a display device 130 without directly interacting 
with expansion module 120, or handheld computing device 1 10. 

It should be understood that system configurations are disclosed here by way 
of example. Other system architectures, platforms, and implementations that can 
support the transmission and display of information on an external display may be 
utilized. Thus, a system and method for displaying graphical presentations on a 
display device using a handheld computing device is described, in conjunction with 
one or more specific embodiments. These and various other adaptations and 
combinations of features of the embodiments disclosed are within the scope of the 
invention. The invention is defined by the following claims and their full scope of 
equivalents. 



-41 - 




jjjjp i 



X X JO 



■> 1 




3 35? 



So 




« « « « '*» « " *"> OOOOQOO Q O 
1Q1HS 
ZCTWS 



I 

U + 

X u 
•0 IhH 

x: J &tj n 

« O J I 

I 1 0,0 

CJ CU IX 

X fa cu E- 

H * fa * 

• • in 

H H (M H 



+ I 

IO iH + T) 

O X TJ X <-» m 

X H,H X £ 

H, IX I H, « 

I DO OT O I I 

00, I I I O, 01 

lO* H 0( IX 

CU fa X fa Oi H 

fa . Eh • fa • 

* vd • r- * 00 

n H T H IT) H 



M 



CO [J 

E- CO II 

co x I o co to o£ to 2 co 2 u >< 
x ko: Q > I I I I I I I CO 2, CO 

K 13 (O U <n,0 OS OC O, O, A3 CO X, I >. 

I DC Ou CM CQ I J I I I I I I I U I 

a:, I ioNWfc<<<<<rfrt;2< 

ICUUQU^E-OOOOOOO^O 
CufaQQQtOO>>>>>>>CO> 

fa • q ■ a • x 

• a> • o • 'CMoror-i^rrginmvD 
vD^r^cvjoocvimcNi^iCNii-tcNii-irvirHCNi 








>« 




1 

1 

o 






i 

1 






1 






| 












| 


| 




1 


Z 




1 

X 


If 






.5 10 




5** 



CM d 



O -H — 1 

H > 



So 



So 



55 



o 

Oh 



rtS'2)0PPAV 
2(S'Z>0PPAV 
itSSJOPPAV CM pj 



2 1{S2)0P«3V 



I 1 



III' i 



L 



5 



Oo 



55 



So 

Od 



5 



Hi- I 



So 



fori" 
>° — hmn 



law aaw mm 

gemggeee pogggopo gags oca a* a 



JSSL. 

OOQOi 

U.IHLU. 



OPPAJ 
IPPAJ 
ZPPAJ 



lOOO 



o 
o 
o 

CM 

9 



«- ft r* « «i o r- 



OQOQOOOO 



ZPUOd 



"tJ 2-2! 
H >11> 



a ItiS ?ss§i 

" O O OOP 

urn 



IP— 





5 ° 



M (4 

c: m 

4 O 

4-» CO 



c - o 

w u u 

« 4-> n 

~0 <n i-> 

C <a u 

u} •-* m 

x a a 



iliil iiiii 



77777 



//////// 



77" 



//////// / 



7777777" 



a oc oo ococ 




o oo 



55 





3 

u 

« V V 

x: - 
i-* i— i 

O «J 

M M O 

0 *J C 
«M C 

_ o >• 

13 O 1 

£ e 

■H D 

> Ji o 

O O rH 

M O OS 

a to 



p c 

p cru 
a v x 
u u 

< M Qi 



I 



§ 

f 



a <u 

O H 

H > 










ZZ 




-«0H- 


— FZ" 


\ 


"TTUH— 




\ 


-TTOH— 


or 


s 


—trnrr" 






— ECW 


— zr 




WJH" 




\ 


— nw— 


— zr 


\ 


3UW 


— BT 




— SOFT - 


— rr 




VCIW 


RT 




ITJH 


58 




dUH 


IV 


N 


IUH 


E» 




— OTH - 


-nsF" 



5 

!»- 



J 



SIOH 
►t(JH 
CiOH 
CIQH 

uoh 

01 OH 
6QH 
9 OH 
/OH 
flOH 
SOM 

rOH 

COM 
ZOH 
tOH 
OOH 



3»A 

lf»A 



8 



(It 

3*fl 

C 



0» 

c 

(4 

a 

CO 

•O 
c 

CO 
X 



I 

W 

•H a 
TJ <d 



5 



DQDDQ 
222S22 



^oSooooooQooooooSoodbQdaQb 




